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Abstract 



This technical report provides data on the extent to which progress monitoring measures 
explain reading achievement. The four progress monitoring measures discussed are: letter 
sounding, phonemic segmentation, oral reading fluency, and comprehension given in either oral 
or written form. These measures were administered to first grade students in low- and high- 
income schools. A regression analysis indicated that the Letter Sounding and Phonemic 
Segmenting tests correlate with each other, as well as the ORF measure, the Listening 
Comprehension test, and the Constructed Response section of the Reading Comprehension test. 
However, these measures do not correlate with the Reading Comprehension test. The ORF and 
Reading Comprehension measures correlated with all other first-grade reading measures. 
Another important finding was that more students from low-income schools needed to take the 
Listening Comprehension test rather than the Reading Comprehension test. This finding 
indicates a potential area of concern for school districts. 
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Introduction 

The No Child Left Behind Act of 2001 has increased the role of assessment in K-12 
education. Designed to ensure that all students meet high academic standards, the law currently 
requires states receiving Title I funds to test all children annually in reading and math in grades 3 
through 8 and report student performance disaggregated by poverty, race and ethnicity, 
disability, and limited English proficiency. By the 2005-06 school year, tests must be expanded 
to include at least one year between grades 10-12, and by 2007-08, states also must include 
science assessments at least once in grades 3-5, grades 6-9, and grades 10-12. The law requires 
states to set annual measurable objectives to track student progress towards reaching proficiency, 
with the ultimate goal that “all groups of students — including low-income students, students 
from major racial and ethnic groups, students with disabilities, and students with limited English 
proficiency — reach proficiency within 12 years” (U.S. Department of Education, 2002, p. 17). 

With this goal in mind, school districts are developing assessment systems that enable 
them to monitor student progress in a timely fashion rather than waiting for year-end statewide 
assessments. These district assessments serve multiple purposes: monitoring student progress, 
evaluating the effectiveness of particular programs and schools, and providing school personnel 
with valuable information about how well their students are doing. Developing easy to 
administer and score assessments at the district level offers schools a distinct advantage over 
depending exclusively on statewide assessments for progress monitoring. In the area of early 
reading, four measures provide essential information about students’ developing proficiency: 
letter sounding, phonemic segmentation, oral reading fluency (ORF), and comprehension given 
either orally or in written form depending on student fluency in oral reading. Taken together, 
these four measures should provide diagnostic information about deficiencies in students’ 
developing reading proficiency, allowing schools to modify their instruction as needed. 
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Methods 

Setting and Subjects 

This report summarizes the spring 2003, first-grade reading achievement data from 29 
different schools in an urban school district in the Pacific Northwest. The original data set 
contained 1209 students, but some students were missing data in some but not all of the 
dependent variable measures, so the total sample size used for analyses varies by measure. 
Design and Operational Procedures 

Dependent variables analyzed in this report include scores from the following measures: 
tests of Letter Sounding (n = 1145), Phonemic Segmenting in = 1150), Oral Reading Fluency 
(ORF) {n =1144), and Comprehension with both selected response (SR) and open-ended 
constructed response (CR) questions that was administered either as a Listening Comprehension 
in = 348) or as a Reading Comprehension (n = 770) test. All measures were performed within 18 
weeks of one another, between January and May during the regular school year. Most students 
completed all tests, while a small group of students completed only a subset due to absence. 
Students who participated in one or more of the assessments were included in the study. Prior to 
analysis, schools in the district were coded into two regions, corresponding roughly with 
household income level. The data set contained 692 students from low-income schools and 465 
students from high-income schools. One school, with a sampled student population of 52, was 
unable to be grouped as either high or low income, so data from these students was not included 
in analysis of variance by income level. Independent blocking variables analyzed in this report 
include school of attendance, income level (as determined by school of attendance), and gender. 
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Measurement/Instrument Development 
Letter Sounding 

The test of Letter Sounding measured students’ ability to recognize letters and make 
appropriate letter sounds. A meta-analysis of letter sounding studies by the National Reading 
Panel (2000) revealed that early letter sounding ability is critical in the development of 
successful readers. Students were given a page containing 80 letters and asked to move across 
the rows, sounding as many letters as possible from that page in one minute. The font used 
(Comic Sans) was selected because its features represent characteristics of common typefaces in 
published materials for children. Trained school district personnel following a standardized, 
written protocol provided by the district administered these tests. 

Phonemic Segmenting 

Phonemic Segmenting tests a student’s ability to hear and recognize phonemes in spoken 
syllables and words. For example, the word hat can be segmented into constituent sounds of /h/ 
/a/ /t/. Strong early readers possess a well-developed phonemic awareness, while the inability to 
segment has been shown to be a good predictor of reading failure (National Reading Panel, 
2000). A meta-analysis of 52 phonemic awareness studies by the National Reading Panel (2000) 
confirmed that teaching phonemic awareness in young children greatly improved the likelihood 
that they would be successful readers. Early recognition of student difficulty segmenting 
phonemes is critical in implementing an intervention to address the difficulty, and thereby set the 
student on a path toward reading success. 

The Phonemic Segmenting test measured students’ fluency in segmenting a series of 
common words. Students were shown a list of 31 words and asked to move across the rows, 
segmenting as many words as possible from that page in one minute. The font used (Comic 
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Sans) was selected because its features represent characteristics of common typefaces in 
published materials for children. Trained school district personnel following a standardized, 
written protocol provided by the district administered these tests. 

ORF 

The test of Oral Reading Fluency was administered individually to each student by 
trained assessors. Students read aloud for exactly one minute one of two comparable passages 
deemed grade-level appropriate on the Fie sch- Kincaid reading scale. At the end of one minute, 
assessors marked the last word read then counted the total words read, subtracting any words 
read incorrectly to arrive at a final ORF score. Self-corrected errors were counted as correct. 
Comprehension 

In addition, first-grade students were administered a Comprehension test using the same 
story they had read for the ORF test. Students who read fewer than 29 words per minute 
correctly on the ORF test completed the Listening Comprehension test. They listened as the test 
administrator read the passage aloud and then read them five multiple choice and two open 
ended questions. Students who read at least 29 words per minute correctly on the ORF test 
completed the Reading Comprehension test. They read the passage aloud and then completed 
five multiple choice and two open ended questions. Each form of the reading comprehension test 
consisted of a reading passage followed by Selected Response (SR) as well as Constructed 
Response (CR) questions. Test administrators who were trained by the district scored student 
responses. All test administrators followed a typed administrating and scoring protocol provided 



by the district. 
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Data Preparation and Analysis 

Student performance on all measures was analyzed using analysis of variance (ANOVA), 
and relationships between key variables were analyzed using correlations and linear regression. 
Alpha was set at .05 for all analyses. 

Results 

A significant difference was found between student performance with regards to gender 
on the test of Phonemic Segmenting, F(l,1148) = 20.15, p < .05, with females outperforming 
males. It should be noted, however, that while this difference was statistically significant, the 
effect size was quite small, accounting for only 2% of the variation in scores on Phonemic 
Segmenting. No other significant differences in student performance blocked gender were found 
on any other measures (see Tables 1 and 2). Significant differences were also found between 
student performances with regards to school income level on the ORF [F(l,1099) = 17.75, p < 
.05]; the test of Letter Sounding [F(l,1094) = 6.33, p < .05]; the SR section of the Reading 
Comprehension test [F(l,814) = 22.28, p < .01]; and the CR section of the Reading 
Comprehension test [F(l,814) = 30.22, p < .01]. In addition, a statistically significant 
disproportionate number of students from low income schools were administered the Listening 
Comprehension test as opposed to the Reading Comprehension test [F(l, 1076) = 22.58, p < 

.01], indicating a statistically significant difference in oral reading fluency by income level of 



school. 
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Table 1 



Descriptive Statistics for Grade 1 Reading Measures 



Group 




n 


M 


SD 


Letter Sounding 


Male 


564 


43.03 


12.37 


(# of letters sounded out 


Female 


581 


43.64 


12.79 


correctly in one minute) 


High Income 


437 


44.30 


12.75 




Low Income 


659 


42.33 


12.58 


Phonemic Segmenting 


Male 


570 


44.77 


14.30 


(# of phonemes read correctly 


Female 


580 


48.60 


14.65 


in one minute) 


High Income 


440 


45.89 


14.82 




Fow Income 


660 


47.33 


14.57 


ORF 


Male 


565 


65.76 


39.12 


(# of words read correctly in 


Female 


585 


69.10 


41.43 


one minute) 


High Income 


442 


72.45 


41.65 




Fow Income 


659 


62.20 


38.12 


Listening Comprehension 


Male 


173 


3.75 


1.31 


Selected Response 


Female 


175 


3.87 


1.29 


(raw score out of 5) 


High Income 


96 


3.75 


1.42 




Fow Income 


245 


3.82 


1.26 


Listening Comprehension 


Male 


173 


2.99 


1.11 


Open Ended 


Female 


175 


3.01 


1.05 


(raw score out of 5) 


High Income 


96 


3.04 


1.03 




Fow Income 


245 


2.98 


1.12 
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Reading Comprehension 


Male 


422 


3.99 


1.67 


Selected Response 


Female 


433 


4.19 


1.46 


(raw score out of 5) 


High Income 


368 


3.79 


1.88 




Low Income 


442 


4.31 


1.25 


Reading Comprehension 


Male 


428 


2.94 


1.42 


Open Ended 


Female 


433 


3.09 


1.28 


(raw score out of 4) 


High Income 


369 


2.75 


1.52 




Low Income 


447 


3.26 


1.15 
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Table 2 



Analysis of Variance Summary Table for Grade 1 Reading Measures 



Source 


df 


F 


T 


P 


Letter 


Gender 


1 


0.67 


.00 


.41 


Sounding 


Error 


1143 


(158.31) 








Income 


1 


6.33* 


.01 


.01 




Error 


1094 


(159.97) 






Phonemic 


Gender 


1 


20.15 


.02 


.00 


Segmenting 


Error 


1148 


(209.65) 








Income 


1 


2.54 


.00 


.11 




Error 


1098 


(215.23) 






ORF 


Gender 


1 


1.98 


.00 


.16 




Error 


1148 


(1624.95) 








Income 


1 


17.75** 


.02 


.00 




Error 


1099 


(1566.11) 






Listening 


Gender 


1 


0.85 


.00 


.36 


Comprehension 

Selected 


Error 


346 


(1.69) 






Response 


Income 


1 


0.18 


.00 


.67 




Error 


339 


(1.70) 
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Listening 

Comprehension 


Gender 


1 


.02 


.00 


.88 


Open Ended 


Error 


346 


(172.29) 








Income 


1 


0.20 


.00 


.66 




Error 


339 


(U8) 






Reading 

Comprehension 


Gender 


1 


3.36 


.00 


.07 


Selected 


Error 


853 


(2.46) 






Response 


Income 


1 


22.28** 


.03 


.00 




Error 


808 


(2.46) 






Reading 

Comprehension 


Gender 


1 


2.45 


.00 


.07 


Open Ended 


Error 


859 


(1.82) 








Income 


1 


30.22** 


.04 


.00 




Error 


814 


(1.77) 







Note. Values enclosed in parentheses represent mean square errors. 
*p < .05, **p < .01. 



For correlational analyses, scores on the comprehension test were separated by those 
students who took the test as a listening comprehension test and those who took it as a reading 
comprehension test. When using data from the Listening Comprehension test, a significant 
correlation between all of the measures was found, with the exception of CR section of the 
Listening Comprehension test. The strongest correlation (r = .37) existed between the Phonemic 
Segmenting and Letter Sounding tests (see Table 3). When using data from the Reading 
Comprehension test, a significant correlation was found between all of the measures. The 
strongest correlation (r = .66) existed between the SR and CR section of the Reading 



Comprehension Test. 
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Table 3 



Correlations Between the Grade 1 Reading Measures, Using Listening Comprehension 







Letter 


Phonemic 


ORF 


Selected 


Constructed 






Sounding 


Segmenting 




Response 


Response 


Letter 


Pearson Correlation 


1 


.371** 


.341* 


.214** 


.056 


Sounding 


Sig. (2-tailed) 




.000 


* 


.000 


.307 










.000 








n 


1145 


1138 


1123 


339 


339 


Phonemic 


Pearson Correlation 




1 


.176* 


.102 


-.049 


Segmenting 


Sig. (2-tailed) 






* 


.059 


.362 










.000 








n 




1150 


1127 


343 


343 


ORF 


Pearson Correlation 






1 


.259** 


.081 




Sig. (2-tailed) 








.000 


.139 




n 






1144 


336 


336 


Selected 


Pearson Correlation 








1 


.216** 


Response 


Sig. (2-tailed) 










.000 




n 








348 


348 


Constructed 


Pearson Correlation 










1 


Response 


Sig. (2-tailed) 














n 










348 



**. Correlation is significant at the .01 level (2-tailed). 
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Table 4 



Correlations Between the Grade 1 Measures, Using Reading Comprehension 







Letter 


Phonemic 


ORF 


Selected 


Constru 






Soundin 


Segmenting 




Response 


cted 






g 








Respons 

e 


Letter 


Pearson Correlation 


1 


.371 


.341** 


.018 


.008 


Sounding 


Sig. (2-tailed) 




.000 


.000 


.601 


.831 




n 


1145 


1138 


1123 


806 


806 


Phonemic 


Pearson Correlation 




1 


.371 


.002 


.125** 


Segmenting 


Sig. (2-tailed) 






.000 


.953 


.000 




n 




1150 


1138 


807 


807 


ORF 


Pearson Correlation 






1 


.204** 


.175** 




Sig. (2-tailed) 








.000 


.000 




n 






1144 


808 


808 


Selected 


Pearson Correlation 








1 


.655** 


Response 


Sig. (2-tailed) 










.000 




n 








855 


855 


Constructe 


Pearson Correlation 










1 


d Response 


Sig. (2-tailed) 














n 










861 



Note: *. Correlation is significant at the .05 level. **. Correlation is significant at the .01 level 
(2-tailed). 



Comparison of Student Performance Based on Oral Reading Fluency 

As expected, students who read the fewest number of words aloud correctly on the ORF 
performed significantly more poorly on all reading measures than students who read at least 29 
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words per minute. Proficiency in ORF accounted for 3-10% of the variance in student 
performance on the other measures (see Table 5). 

Table 5 



Analysis of Variance Summary Table Comparing Students Who Took the Listening 
Comprehension Test with Those Who Took the Reading Comprehension Test 



Source 




df 


F 


2 

11 


P 


Letter Sounding 


Group 


1 


100.85** 


.08 


.00 




Error 


1097 


(142.13) 






Phonemic Segmenting 


Group 


1 


42.37** 


.04 


.00 




Error 


1101 


(201.11) 






ORF 


Group 


1 


802.60** 


.42 


.000 




Error 


1096 


(986.41) 






Comprehension 
Selected Response 


Group 


1 


114.91** 


.10 


.00 




Error 


1086 


(0.89) 






Comprehension Open 


Group 


1 


32.32** 


.03 


.00 



Ended 

Error 1087 (0.83) 

Note. Values enclosed in parentheses represent mean square errors. 

*p < .05, **p < .01. 

Regression Analysis of District Reading Assessments 

Because students do not take the statewide large-scale assessment until the spring of their 
third-grade year, district assessment of student progress in reading becomes even more important 
in the early years. At the end of their third-grade year, students’ reading ability is assessed using a 
SR reading comprehension test administered by the state. The measure administered by the 
district that most closely resembles the statewide test is the SR section of the Comprehension test. 
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A linear regression model in which student performance on Letter Sounding, Phonemic 
Segmenting, ORF, and CR Listening Comprehension were used to predict performance on the SR 
Listening Comprehension test accounted for 10% of the variation on the SR Listening 
Comprehension test [F( 4,322) = 8.60, p < .05]. Table 6 presents the regression analysis using the 
SR Listening Comprehension test as the dependent variable. 

A linear regression model, in which student performance on Letter Sounding, Phonemic 
Segmenting, ORF, and CR Reading Comprehension was used to predict performance on the SR 
Reading Comprehension test, accounted for 28% of the variation on the SR Listening 
Comprehension test [F(4,789) = 77.74, p < .05]. Table 7 presents the regression analysis of the 
SR Reading Comprehension test as the dependent variable. 

Table 6 

Regression Summary for Grade 1 SR Listening Comprehension Test 



Independent Variables 


Unstandardized 

Coefficients 


Standardized 

Coefficients 




95% Confidence Interval for B 


B 


Std. 

Error 


Beta 


t 


Lower Bound 


Upper Bound 


Letter Sounding 


1.E- 


.01 


.13 


1.95 


.00 


.03 




02 












Phonemic Segmenting 


2.E- 


.01 


.02 


0.34 


-.01 


.01 




02 












ORF 


2.E- 


.01 


.16 


2.70 


.01 


.04 




02 












District Listening Comprehension (CR) 


0.20 


.06 


.17 


3.07 


.07 


32 


Constant 


2.12 


.33 




6.53 


1.48 


2.76 
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Table 7 

Regression Summary for Grade 1 SR Reading Comprehension Test 



Independent Variables 


Unstandardized 

Coefficients 


Standardized 

Coefficients 


t 


95% Confidence Interval for B 


B 


Std. 

Error 


Beta 


Lower Bound 


Upper Bound 


Letter Sounding 


2.E-03 


.00 


.02 


0.71 


-.00 


0.01 


Phonemic Segmenting 


***** 


.00 


-.07 


-2.28 


-0.01 


-0.00 


ORF 


4.E-03 


.00 


.10 


3.12 


.00 


0.01 


District Reading Test (CR) 


0.58 


.04 


.51 


16.66 


.51 


0.64 


Constant 


2.36 


.22 




10.92 


1.94 


2.79 



Discussion 

District Letter Sounding Test 

The District Letter Sounding test is significantly correlated with performance on first- 
grade Phonemic Segmenting, ORF, and the CR section of the District Reading Comprehension 
test. While it provides information that can be used by means of a regression equation to predict 
student performance on the Listening Comprehension test, it does not serve this same purpose 
for the Reading Comprehension test. Like the Phonemic Segmenting test, the Letter Sounding 
test will be most useful to the district as a screening mechanism to allow teachers to group 
students appropriately for instruction and to enable schools to allocate additional resources to 
ensure that all students are making adequate progress in terms of their reading development. 
District Phonemic Segmenting Test 

The District Phonemic Segmenting test is significantly correlated with performance on 
first-grade ORF, Letter Sounding, and the CR section of the District Reading Comprehension 
test. It does not, however, provide useful information that can be used by means of a regression 



equation to predict student performance on the Reading and Listening Comprehension tests. It 
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will be most useful to the district as a screening mechanism to allow teachers to group students 
appropriately for instruction and to enable schools to allocate additional resources to ensure that 
all students are making adequate progress in terms of their reading development. 

ORF 

The ORF is significantly correlated with all other first-grade reading measures 
administered by the district. It provides useful information that can be used by means of a 
regression equation to predict student performance on the SR section of the Reading and 
Listening Comprehension tests. The ORF is easy to administer and does not require much time 
or training to score; it can continue to be a useful source of information for teachers monitoring 
student growth in reading. 

District Comprehension Tests 

Both the Listening and Reading Comprehension tests provide useful information to the 
District about the progress students are making in reading. The disproportionate number of 
students whose scores on the ORF indicated a need to administer the Listening Comprehension 
test as opposed to the Reading Comprehension test from schools identified as serving low- 
income students should alert the district to a potential area of concern. An aggressive focus on 
ensuring that all students are making adequate progress in developing reading skills regardless of 
income level or school of attendance is recommended. 

The district’s current reading assessment kit can offer insights into strengths of particular 
programs, schools, and teachers and provides school personnel with information that can help 
them measure student progress towards reading proficiency. It will continue to be revised, and 
the revisions will be analyzed using Item Response Theory (IRT) in subsequent years as the 
district works to improve the reliability and validity of the instruments for the various ways they 
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are used. Additional technical reports will be written to follow up on these analyses and 
document the changes being made to the reading assessment kit. 
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